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This summary of a presentation made at the Adult 
Education Research Conference in Minneapolis, Minnesota, February 
27-28, 1970 deals with an evaluation research project undertaken at 
the Remscheid Academy for Free Creative Art Expression and Media 
Education. This institution aims at education and training of present 
or future adult educators and youth leaders. Course participants, who 
are between 18 and 26 years of age, are instructed by seven trained 
artists. Evaluation was attempted through a series of measures of 
attitude change, to determine the effect of three short courses and 
one semester-length course in introducing a positive approach towards 
a progressive type of art education. A Thurstone- and Likert-type 
rating scale were used, as well as a 20 item multiple choice test. A 
more extensive report of the research is to be published by the Adult 
Education Department, University of Leuven (Louvain) , Belgium. (MF) 
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LEIRMAN, UALTER bvaxuaiiua oj. au.nuue cnange enrough creative art and 
expression courses in Remscheid (W. Germany). 

The present abstract briefly summarizes the Setup and the results 
of an evaluation research project carried out in the "Akademie Remscheid 
fur Musische Bildung und Medienerziehung" , i.e. the Remscheid Academy 
for free creative art expression and media education . This institution, 
which aims at the education and training of present or future youth leaders 
and adult educators and in which seven trained artists instruct and 
animate course participants between 18 and 26 years of age, is of its 
kind unique in Europe. 

After a pilot-study in August 1968, where we used a before-after 
measurement technique, complemented by participant observation in each 
of six workgroups, the research project started with courses Nrs 130 and 
131 in April 1969. The project will cover one year of course-work. 

So far, three short "basic" courses (3-4 weeks) with a total of 114 
participants and one "semester-course" (4 months) with 23 participants 
have been investigated, and the larger part of the results are available. 

A first extensive report titled Musische Bildung in Remscheid I. Eine 
Ergebnisunter suchung , will soon be published by the Adult Education 

r* 

Department of the University of Leuven (Belgium) to which the author 
belongs. The general design consists of what we have called an ANTE - 
META - POST measurement, involving Ante- Meta- Post questionnaires and 
a specially constructed Ante - Meta test. 

Attitude change was measured through a Thurstone - and Likert-type 
rating scale and a multiple-choice question containing 20 items. These 
two tasks were administered without any change immediately before (ANTE), 

immediately after (META) and three-four months after the course (POST). 
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Changes in attitude were tested by use of such distribution-free tests 
as Wilcoxcon's matched-pairs singned-rank test and the sign- test. 

The results were compared to those obtained from a control-group on 
the same research instruments. 

Thus far, the three "basic" courses have shown a positive change in 
attitude towards a progressive type of art education directly after the course 
(META) whereas the longer semester course showed a negative change in 
attitude away from the traditional, somewhat romantic concept of creative 
art education. However, the only POST - measurement carried out on the 

first basic course shows no significant change when compared to the ANTE- 

$ 

measurement. Apparently, the course-effect was not strong enough to make 

the change permanent. The test specially constructed for our study, 

called TELOTEST, aimed at evaluating the change in participation goals 

(Gr. telos = goal) from ANTE to META through POST. 

In ANTE and META, it consisted of a restricted or balanced incomplete 

2 

pair-comparison design (Mos teller's X -test for goodness of fit was applied 
with good results in every case), a positive-negative preference rating 
(sign- test), and a goal-discussion test (t - test and analysis of variance). 
In POST, where the only instrument available was a mailed quest ionnaire$ 
we only used the positive-negative preference rating. The results indicate 
that the goals with either high or low ANTE-ratings were generally not 
significantly affected by the course, whereas the intermediate goals 
either gained or lost significance. On this test, the control-group 
again showed almost no significant change. Repetition of this TEIX)TEST in 
other settings, withoother kinds of items, could lead to further refinement 
of this technique. Full details, including tables, will be given in the 
paper . 
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THEORY AND METHODS OF EVALUATION RESEARCH: 

THE REMSCHEID ACADEMY. 

Dr. Walter Leirman 
Univ. of Leuven (Belgium) 

The present abstract briefly summarizes the general organization and the results 
of an evaluation research project, carried out in the "Akademie Remscheid fu r 
Musische Bildune und Medienerziehung" , i.e. the Remscheid Academy for creative 
art expression and media education . This institution aims at the education and 
training of present or future youth leaders and adult educators; in it, seven 
trained artists instruct and animate course participants between 18 and 26 
years of age (generally). It is of its kind unique in Europe. 

After a pilot-study in August 1968, where we used a before-a: ter measurement 
technique, complemented by participant observation in each of six work-groups by 
9 of our students, the present research project started with courses Nrs 130 and 
131 in April 1969. The whole project will cover one year of course work. 

So far, three short basic courses (3-4 weeks) with a total of 114 participants, 
and one semester-course (4 months) with 23 participants, have been investigated, 
and the larger parts of the results are now available. A first extensive report, 
titled Musische Bildung in Remscheid I. Eine Ergebnisuntersuchung will soon be 
published by the Adult Education Department of the University of Leuven (Belgium), 
to which the author belongs. The general design consists of what we have called 
an ANTE-META-POST measurement, involving Ante-, Meta- and Post-questionnaires, and 

a specially constructed Ante-Meta-Post test. 

Attitude change was measured through a Thurstone— and Likert-type rating scale 
concerning four current definitions of creative expression education, and a 
multiple-choice question containing 20 word-items. The four definitions represented 
four different tendencies, from the romantic— reformist to the progressive-renewal 
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concept. This classification was done by four independent judges. These two tasks 
were administered with almost no change, immediately before (ANTE), immediately 
after (META) and three-four months after the course (POST). Changes in attitude 
were tested by such distribution-free tests as Wilcoxon's matched-pairs signed-rank 
test, (Vs) and the sign-test. The results were compared to those obtained from a 
control— group on the same research instruments. Thus far, the three basic 
courses have shown a positive change in attitude towards the progressive type of 
art education advocated by most of the instructors, directly after the course 
(ANTE-META comparison). The longer semester-course showed a negative change, in 
that it significantly moved away from its ANTE-ratings of the traditional, somewhat 
romantic concepts of creative art education. However, the only POST-measurement 
carried out on the first basic course, shows no significant change when compared 
to the ANTE-results (ANTE-POST comparison). Apparently, the course-effect was not 
strong enough to make the change permanent. In the meantime, the control group 
showed no change at all. The results obtained from the multiple choice task 
confirmed the same trend. 

The test specially constructed for our study, called TELOTEST, aimed at 
evaluating the change in participation-goals (Gr. telos = goal) from ANTE to MEIA 
through POST. 

In ANTE and META, it consisted of 

(1) a restricted or balanced incomplete pair comparison design containing 
13 goal statements, written in behavioral terms 

(2) a positive-negative preference rating of the same 13 goal statements 

(3) a goal-discussion test of 20 statements (including the original 13) 
with groups of 4 participants. 

The significance tests applied were. 




- 3. - 






•for test item (1) : Mostseller's Chi-square-test for goodness of fit 

(with positive results in every case thus far) 

for test item (2): the sign-test 

for test item (3): t-tests per statement, Kruskall-Wallis analysis of 

variance for all courses. 

In POST, where the only instrument available was a mailed questionnaire, we 
only used the positive-negative rating (item 2). Results of the 3 items were 
correlated (Spearman-rank test), so as to gain information about the validity of 
the test. We obtained very high correlations for items 1 and 3 in every case, 
where as the coefficients obtained from correlating item 2 with the two other 

ones varied from -.32 to *t91. 

~ ANTE-META comparisons of Telotest (1) and (2) indicate that the goals with 

either high or low ANTE-ratings were generally not significantly affected by the 
course, whereas the intermediate goals either gained or lost significance. On 
this test, the control-group again showed almost no change. 

In a forthcoming publication, the discussion of this specific research project 
will be preceded by two chapters on theory (models, systems and units) and design 
of evaluation research. The following chart briefly illustrates how we tried to 
link theory and practice in one project. 
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REMSCHEID STUDY DESIGN CHART: MEASUREMENTS 






